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Introduction 

There  are  several  general  forms  of  expression  that  people  can  use  in 
generating  instructions  for  themselves,  other  people,  or  machines  to  follow. 
These  include  forms  emphasizing  a description  of  a desired  object  or  state 
(e.g.,  "Get  a list  of  employees  over  forty  years  of  age  who  have  five  or 
more  dependents")  and  forms  emphasizing  a procedure , or  sequence  of  steps 
to  be  carried  out,  (e.g.,  "Go  to  the  employee  file.  Take  out  the  first 
folder.  If  the  employee  is  over  forty  then...") . Instructions  may  be  given 
in  ordinary  English  or  in  cin  artificial  language  with  a restricted  syntax, 
such  as  a computer  programming  language  or  the  special  notation  used  in  knit- 
ting instructions.  Besides  restricted  syntax,  these  artificial  languages 
have  a restricted  lexicon,  a restricted  semantic  domain,  and  specialized 
conventions . 

In  designing  computing  systems  to  be  used  by  non-programmers,  questions 
arise  about  the  merits  of  various  forms  for  expressing  instructions  to  the 
computer.  Are  descriptions  more  "natural"  than  prodecures?  Do  instructions 
expressed  in  a restricted  syntax  Icinguage  take  longer  to  write  than 
instructions  expressed  in  English?  Do  untrained  persons  prefer  one  form 
of  expression  over  others?  Which  approach  is  most  accurate?  Does  an 
individual  use  the  same  approach  in  related  task  environments?  Can  task 
environments  be  studied  to  determine,  to  a useful  degree,  the 
characteristics  of  people's  linguistic  productions?  The  experiments 
reported  below  were  designed  to  permit  these  and  similar  comparisions  among 
various  forms  of  expression  used  by  subjects  on  some  simple  tasks. 

In  the  first  experiment  we  asked  subjects  to  write  various  kinds  of 
instructions  and  observed  the  time  taken  to  write  the  instructions  and 
the  adequacy  of  the  instructions  produced.  In  one  condition  subjects 
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were  not  constrained  to  use  any  particular  form  of  expression;  we  were 

interested  in  the  forms  they  would  choose.  In  other  conditions,  subjects 

were  required  to  write  procedural  instructions,  descriptive  instructions, 

or  use  a simple  restricted  syntax  language.  To  provide  a practical  test 

of  the  instructions  produced,  each  subject  was  given  the  instructions  , 

written  by  another  subject,  and  she  was  then  required  to  carry  them  out. 

For  comparison,  subjects  also  were  asked  to  carry  out  instructions  carefully 
prepared  by  experimenters.  Speed  and  accuracy  of  performance  were  recorded. 

In  the  second  experiment  we  concentrated  on  subjects ' preferences  for 
forms  of  expression.  We  looked  again  at  unconstrained  expression,  and 
also  asked  subjects  to  rate  the  naturalness  and  effectiveness  of 
instructions  of  various  kinds. 

Experiment  I 

Method 

Subjects . Subjects  were  fourteen  general  filing  clerks  employed 
by  a local  temporary  manpower  agency.  They  were  generally  middle  aged 
(median  = 44;  range  21  - 63)  high  school  graduates  (one  had  only  two 
years  of  high  school  and  six  had  some  education  beyond  high  school) 
who  sometimes  worked  part-time  outside  the  home. 

Design  and  Procedure.  Two  task  settings  were  used.  In  Blocks 
sxibjects  wrote  protocols  (this  term  refers  to  the  material  that  subjects 
wrote)  about  figures  made  of  colored  cubes  (children's  blocks).  Fourteen 
figures  were  used,  including  a horizontal  row  of  eiqht  blocks,  a double 
staircase  of  twenty  blocks,  and  a figure  with  a jagged  base  and  irregular 
height  containing  13  blocks.  In  Typing  subjects  wrote  protocols  about 
6x6  figures  of  characters.  Seven  figures,  all  composed  of 
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X's  and  blanks,  and  all  having  simple  verbal  descriptions,  were  used.  For 
example,  one  figure  had  odd-numbered  columns  filled  with  X's  and  cinother 
had  a diagonal  of  X's. 

For  each  task  setting  a simple  restricted  syntax  language  was  devised. 

For  Blocks,  subjects  could  write 

START  WITH  

(Block  color) 

followed  by  a series  of  statements  of  the  form 

PUT  

(block  color)  (spatial  relation,  (block  color) 

e.g.  "to  the  right 
of",  "above) 

All  figures  were  designed  so  that  they  could  be  described  unambiguously  in 

this  language.  The  typing  language  had  only  one  statement: 

HIT  . 

(key,  i.e.,  (times,  e.g.  3) 

X,  space, 
return) 

Subjects  were  required  to  generate  instructions  and  to  follow 

instructions.  In  generating,  a subject  was  given  a figure  and  told  to 

write  a protocol  of  a particular  kind.  For  example,  she  was  told,  "Please 

write  a detailed  description  of  this  figure."  Blocks  figures  were  placed 

in  front  of  the  subject  for  these  trials,  while  Typing  figures  were  only 

(4) 

described  orally  to  her.  In  following,  a subject  was  given  a protocol 

written  by  another  subject  and  asked  to  build  a figure  from  it. 

Subjects  wrote  protocols  under  each  of  four  different  constraints, 
conveyed  by  the  instructions  of  the  experiment.  Under  the  Description 
constraint  a subject  was  told,  "Please  write  a detailed  description  of 
this  figure."  Instructions  for  the  Procedure  constraint  were,  "Please 
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write  a procedure,  or  plan,  to  build  [or  typel  this  figure."  Under  the 
Restricted  Syntax  constraint  a subject  was  taught  the  restricted  syntax 
language  for  the  task  and  told  to  use  it.  The  Neutral  constraint 
instructions  were  intended  not  to  restrict  the  form  of  the  protocol.  They 
were,  "Please  describe  in  writing  how  you  would  tell  somebody  else  how  to 
build  or  type  this  figure."  As  part  of  the  instructions  for  each 
constraint,  subjects  were  told  that  they  must  be  specific  cibout  "the  color 
and  position  of  each  block"  in  blocks  (there  were  four  colors) , auid  "each 
filled  and  unfilled  character  space"  in  typing. 

The  sequence  of  events  for  a typical  subject  is  shown  in  Figure  1. 

The  ordering  of  events  within  brackets  was  balanced  across  subjects.  Notice 
that  every  subject  had  one  Neutral  constraint  trial  before  learning  the 
restricted  syntax  language  or  seeing  experimenter-written  examples  of  it, 
and  a second  Neutral  constraint  trial  afterwards.  Except  for  the  variations 
within  the  brackets,  the  order  shown  in  Figure  1 was  used  for  all  subjects; 
all  subjects  worked  on  Blocks  before  typing.  In  following  protocols 
written  by  another  subject  (8th  through  12th  events  in  Figure  1) , each 
subject  followed  those  written  by  the  previous  subject  (a  pilot  subject  was 
used  for  the  first  subject) . The  fourteen  Blocks  figures  used  were 
divided  into  two  sets  of  seven  which  were  given,  respectively,  to  odd  and 
even-numbered  subjects,  so  that  no  subject  would  be  following  a protocol 
for  a figure  she  had  already  seen,  within  these  sets,  the  assignment  of 
figures  to  events  was  balanced  across  subjects. 

Although  subjects'  performance  was  timed  with  a stopwatch  by  an 
experimenter,  they  were  told  they  were  not  under  time  pressure. 
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Results 

Generating . The  times  for  protocol  generating  are  shown  in  Table  1. 
There  were  no  large  differences  in  the  times  required  to  write  under  the 
four  constraints  (F  tests,  all  .10).  Subjects  rarely  erased,  crossed 

out,  or  made  insertions,  and  they  rarely  reviewed  what  they  wrote. 

To  assess  the  quality  of  subjects'  output  the  three  experimenters 
independently  classified  each  protocol  as  ambiguous  or  unambiguous  and, 
separately,  as  consistent  or  inconsistent.  A protocol  classified  as 
eimbiguous  and  consistent  is  one  which  (a)  specifies  more  than  one  figure, 
and  (b)  one  of  which  is  the  target  figure.  A protocol  classified  as  un- 
ambiguous and  inconsistent  is  one  which  (a)  uniquely  specifies  just  one 
figure,  but  (b)  which  is  not  the  target  figure.  Table  2 shows  the  results 
of  this  classification. There  is  not  much  difference  in  consistency 
under  the  various  constraints,  i.e.  about  65  - 80%  of  protocols  were  con- 
sistent with  the  target  figure  under  each  constraint.  As  might  be  expected. 
Restricted  Syntax  protocols  were  less  often  ambiguous  than  the  others 
(28%  vs.  50%) . The  second  Neutral  constraint  protocols  were  also  less 
frequently  ambiguous  than  the  others,  mainly  because  these  protocols  showed 
the  influence  of  the  Restricted  Syntax  language. Subjects  readily 
adopted  the  succinct,  precise  shorthand  notations  of  the  experimenters. 

The  two  tas)c  settings  seemed  to  ma)ce  different  demands  on  subjects. 

Mainly,  there  were  many  fewer  ambiguous  Typing  protocols  than  Blocl<s 

protocols  (26%  vs  69%).  Necurly  all  these  Bloc)cs'  protocols  contained 

(7) 

ambiguities  about  the  spatial  position  of  the  bloc)cs  . In  Typing,  explicit 
specification  of  the  spatial  position  was  usually  not  required.  Com- 
parisons among  constraints  look  roughly  the  same  for  both  tasks,  however. 
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Individual  subjects  varied  over  a wide  range  in  accuracy.  Of  the  twelve 

protocols  they  wrote,  two  subjects  wrote  eight  unambiguous-consistent  protocols 

and  two  wrote  only  one  such  protocol.  Subjects  were  not  reluctant  to  write 

protocols  (as  we  thought  they  might  be) , although  in  Typing  a few  did  state 

they  knew  how  to  type  a figure  but  not  how  to  write  about  typing  it. 

The  protocols  were  also  classified  by  the  experimenters  as  Procedure- 

(8) 

like  or  Description-like  according  to  their  linguistic  characteristics. 

A protocol  was  classified  as  a Procedure  if  it  contained  sequential 
executable  steps,  action  verbs  or  imperative  mood.  It  was  classified  as  a 
Description  if  it  referred  to  major  components  of  the  target  figures' 
structure,  was  in  the  passive  voice,  or  used  existential  verb  constructions. 

In  addition,  protocols  which  showed  clear  similarity  to  one  of  the  Restricted^ 
Syntax  languages  were  classified  as  Restricted  Syntax.  Table  3 shows  the 
results  of  this  classification.  Most  protocols  (81%)  were  linguis- 
tically characteristic  of  the  expressional  constraints  under  which  they 
were  written;  that  is,  subjects  could  follow  instructions  and  readily  change 
their  mode  of  expression.  Most  Neutral  constraint  protocols  and  even 
several  Description-constraint  protocols  were  classified  as  Procedure-like, 

Many  of  the  second  Neutral  constraint  protocols,  written  after  exposure  to 

the  Restricted  Syntax  language,  incorporated  the  Restricted  Syntax  language, 

(9) 

and  were  so  classified. 

Following.  Times  required  to  follow  experimenter-written  protocols  are 
shown  in  Table  4.  A_  might  be  expected,  following  is  faster  than  writing  (cf. 
Table  1),  (F  (1,13)  ■»  22.16;  P < .001).  Table  4 also  shows  the  accuracy 


Table  4.  Time  And  Accuracy  In  Following  Experimenter-Written  Blocks 
And  Typing  Protocols  In  Experiment  1.  Each  Cell  Is  Based 
Upon  14  Subjects. 


Table  5.  Accuracy  Of  Following  Experimenter-Written  And  Consistent 
Subject-Written  Blocks  Protocols  In  Experiment  1. 


No.  Target  Figures  Correctly  Constructed  31  Id 


No.  Attempts  I 

.59  .04 


Percent  Correct 


12 


Table  6.  Accuracy  Of  Subjects'  Following  The  53  "Consistent" 
Subject-Written  Protocols  (i.e.,  Tliose  53  That 
Correctly  Described  The  Target  Figures,  Among  Others) 
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with  which  the  experimenter-written  protocols  were  followed.  Notice  that 
subjects  were  far  from  perfect  in  following  these  carefully  prepared 
protocols.  Errors  fell  mainly  into  three  classes:  in  Blocks,  skipping 

an  instruction  or  confusing  spatial  relations;  in  Typing,  inserting  extra 
rows  or  columns.  As  shown  in  Table  5,  this  same  level  of  accuracy  resulted 
when  subjects  followed  protocols  written  by  other  subjects  which  were 
consistent  with  the  target  figures  even  though  these  subject-written 
protocols  were  often  ambiguous.  Apparently  subjects  were  able  to 
disambiguate  these  orotocols  using  contextual  cues  and  their  knowledge  of 
the  task  setting.  As  stated  adjove,  the  block  figures  that  subjects  recon- 
structed in  following  were  similar  but  different  from  those  they  had 
previously  worked  with. 

Accuracy  in  following  subject-written  protocols  was  not  dramatically 
affected  by  the  instructional  constraint  under  which  they  were  written 
(Table  6a)  or  gross  linguistic  characteristics  of  the  protocol  (Table  6b)  , 
although  procedural  protocols  seem  to  have  scxne  advantage  over  descriptive 
ones  . 

Discussion 

Two  main  conclusions  emerge  from  the  generation  results.  First,  the 
speed  and  accuracy  of  generating  protocols  does  not  seem  to  depend  much 
upon  the  instructional  constraints  imposed.  The  task  environ.ment  of  these 
two  relatively  simple  tasks  greatly  affects  accuracy,  howe.’er.  Second, 
under  the  Neutral  constraint  subjects  tend  to  choose  procedure-like 
expressions  — in  fact,  they  even  adopted  features  of  the  procedural  Re- 
stricted Syntax  language.  We  had  thought  that  procedures  might  be  followed 
more  quickly  aind  perhaps  more  accurately  than  descriptions  because  a person 
does  not  have  to  generate  his  own  sequence  of  operations  prior  to  performing 
them,  but  this  was  not  the  case  with  these  experimenter-written  protocols. 
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In  compajing  generating  and  following,  it  is  interesting  to  note  that 
while  building  a figure  from  a protocol  is  faster  than  writing  a protocol 
for  that  figure,  the  accuracy  of  the  two  processes  is  about  the  same  in  the 
sense  that  about  75%  of  Subject-generated  Blocks  protocols  were  consistent, 
whereas  64%  of  experimenter-written  Blocks  protocols  were  followed  correctly. 
Why  should  this  be,  given  that  subjects  have  much  more  experience  following 
protocols  (assembly  instructions,  recipes)  than  writing  them?  It  may  be  that 
subjects  were  less  careful  than  usual  while  following  protocols  in  this  ex- 
periment. Even  though  they  were  able  to  disaunbiguate  protocols  in  these 
task  settings,  perhaps  subjects  normally  rely  even  more  on  their  knowledge  of 
task  settings  to  guide  themselves  toward  their  goals.  On  the  other  hand,  home 
economics  teachers  tell  us  that  they  observe  error  rates  at  least  this  high 
when  students  and  adults  attempt  to  follow  cookbook  recipes. 

The  tendency  of  subjects  to  write  procedure-like  protocols  under  the 
Neutral  constraint  has  at  least  two  interpretations,  both  of  which  may  be 
correct.  One  is  that  subjects  have  a strong  tendency  to  write  procedure- 
like protocols  which  they  generally  express  unless  prevented  from  doing  so. 
Another  is  that  the  Neutral  constraint  as  stated  was  not  really  Neutral; 
it  encouraged  subjects  to  write  procedural  protocols.  Experiment  2 
investigates  the  issue  and  attempts  to  discover  subjects'  own  views  on  the 
desirability  of  various  forms  of  expression. 

Experiment  2 

Method 

Subjects . Subjects  were  eleven  female  and  one  male  liberal  arts 
college  students  with  little  or  no  experience  in  using  computer  systems. 
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Procedure . Experiment  2 consisted  of  two  phases,  writing  and  ranking, 
carried  out  in  that  order  by  all  subjects.  In  the  writing  phase  each  subject 
was  asked  to  write  a protocol  for  each  of  three  Blocks  figures.  Half  of 
the  subjects  were  given  the  Neutral  constraint  instructions  used  in 
Experiment  1:  "Please  describe  in  writing  how  you  would  tell  somebody  else 

how  to  build  this  figure.  You  must  be  specific  about  the  color  and  position 
of  each  block."  The  other  subjects  were  told,  "Please  write  how  you 
would  describe  this  figure  to  a friend.  You  must  be  specific  about  the 
color  and  position  of  each  block."  It  was  felt  that  this  instruction 
might  encourage  production  of  description-like  protocols. 

In  the  ranking  phase  each  subject  was  given  a set  of  five  protocols 
tciken  from  Experiment  1 and  asked  to  rank-order  them  according  to  one  of 
the  six  specified  criteria  shown  in  Table  7.  All  protocols  within  a set 
described  a sinqle  Blocks  figure,  placed  in  front  of  the  subject.  The 
five  protocols  to  be  ranked  were  a natural-language  Procedure 
protocol  written  by  a subject  in  Experiment  1,  a Description  protocol  and 
Restricted  Syntax  protocol  from  the  same  source,  and  a Description 
protocol  and  Restricted  Syntcix  protocol  written  by  the  experimenters.  The 
subject-written  protocols  were  selected  from  among  those  rated  unambiguous- 
consistent  in  Experiment  I or  were  revised  slightly  to  bring  them  to  that 
standard.  Each  subject  ranked  six  different  sets  of  protocols,  one  for 
each  of  the  six  criteria  of  Taible  7.  Each  set  of  protocols  was  about 
a different  Blocks  figure.  The  order  in  which  criteria  were  used  in 
ranking,  and  the  pairing  of  figures  and  criteria,  were  balanced  over 
subjects,  except  that  the  three  "ordinarily"  criteria  (see  Table  7) 
always  preceded  the  three  "best  way"  criteria. 
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Results  eind  Discussion 

Writing . As  in  Experiment  1,  the  "Neutral"  constraint  instructions 
produced  mostly  procedure-like  protocols  (see  Table  8) . College  students 
did  not  differ  much  from  middle-aged  file  clerks  in  this  regard.  The 
description-biased  instructions,  on  the  other  hand,  produced  only 
description-like  protocols.  Thus  the  choice  of  form  of  expression  by 
subjects  is  easily  controlled  by  instructions  which  do  not  specifically 
dictate  the  form  to  be  used.  This  does  not  mean  that  subjects  have  no 
general  tendency  to  generate  procedure-like  protocols;  it  only  means 
that  this  tendency  can  be  counteracted  by  fairly  mild  instructions.  The 
different  effects  of  the  two  sets  of  instructions  might  arise  in  two  ways. 

It  may  be  that  subjects  felt  that  the  purpose  for  which  the  protocols  were 
to  be  constructed  was  different  under  the  two  instruction  sets,  aind  chose 
the  most  effective  form  of  expression  in  each  situation.  In  this  case, 
choice  of  form  of  expression  depends  on  the  particular  task  pursued.  On 
the  other  hand,  it  may  be  that  subjects  saw  no  important  difference  in  the 
purpose  of  the  protocols  but  interpreted  the  instructions  as  demanding  a 
particular  form  of  expression.  In  this  case  some  limit  may  be  placed  on 
the  strength  of  preference  in  form  of  expression  subjects  may  have,  since 
the  instructions  made  no  direct  demand. 

When  these  protocols  were  examined  for  ambiguity  and  consistency,  as 
in  Experiment  1,  those  written  under  the  "Neutral"  constraint  were  less 
often  ambiguous  than  those  written  under  the  Description-Biased  instructions, 
as  shewn  in  Table  9.  This  is  consistent  with  the  finding  in  Experiment  1 
that  procedure-like  protocols  were  less  often  cunbiguous  than  description- 


like protocols. 
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Ranking.  Table  10  shows  the  average  ranking  of  protocols  by  type.  The 
"ordinary"  and  "best  way"  criteria  did  yield  different  rankings.  Subject- 
written  procedures  and  descriptions  were,  over-all,  most  preferred; 
restricted  syntax  protocols  were  least  preferred.  The  purposes  specified  in 
the  criteria  did  affect  rankings,  however,  (Purpose  x Protocol  Type  inter- 
action F^(8,88)  = 3.16  £ < .01.)  subject-written  procedures  were  given  highest 
rankings  under  the  general  "to  know  about  the  figure"  criterion.  Descrip- 
tions were  preferred  under  the  "to  identify"  criterion.  As  far  as  these 
s'objects  were  concerned,  then,  there  is  no  single  preferred  form  of 
expression.  Rankings  such  as  these  may  have  limited  validity  in  predicting 
how  people  actually  generate  instructions:  the  experimenter-written 
Restricted  Syntax  protocols  were  given  low  rankings,  while  in  Experiment  1 
subjects  frequently  chose  to  .use  features  of  the  Restricted  Syntcix  language 
in  generating  protocols. 

Summary  and  Conclusions 

The  key  results  of  Experiment  2 are  two.  First,  seemingly  small 
variations  in  instructions  can  lead  to  large  variations  in  the  form  of 
expression  that  people  use,  even  though  the  same  goal  is  sought  in  all  cases. 
Subjects  easily  varied  the  surface  structures  of  their  protocols.  Second, 
when  asked  their  preference  for  protocols  using  different  forts  of 
expression,  subjects  showed  different  preferences  under  different  task 
demcinds . 

The  significance  of  all  the  findings,  based  upon  simple  tasks,  to  the 
designers  of  computer  systems  for  non-programmers  is  that  there  is  no  single 
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"natural"  form  of  expression.  Mild  variations  in  task  environment  and 
expressional  constraints  lead  to  larqe  variations  in  the  way  subjects  express 
themselves.  In  desiqninq  a hypothetical  computer  system  dealinq  with 
Blocks  fiqures,  for  example,  one  could  not  just  find  the  "natural"  way  of 
referrinq  to  them  and  implement  it,  since  there  is  not  a sinqle  natural  way. 
Human  linouistic  and  coqnitive  systems  are  better  characterized  as  adaptive 
than  as  havinq  stronq  "natural  tendencies."  Discoverinq  "natural  ways" 
people  think  and  speak  has  ethological  value  and  sometimes  psychological 
value,  but  may  have  little  value  for  systems  design. 

The  acceptance  of  restr icted-syntcix  languages  by  subjects  strikes  an 
encouraging  note.  Whereas  they  generally  showed  low  preference  for  them 
as  a way  of  expressing  instruction  (Experiment  2) , when  called  upon  to 
exoress  instructions  themselves,  subjects  readily  and  sometimes  spontaneously 
used  restricted  syntax  (Experi.ment  1)  . So  it  may  not  always  be  necessary  or 
desiraible  to  avoid  such  language  restrictions  in  system  design,  even  when 
it  is  possible  to  do  so.  Well  designed  restricted-syntax  languages  that  map 
well  onto  a problem  can  provide  thought- templates  for  a person  describing 
or  expressing  that  problem.  There  are  major  distinctions  bet’^^een  a "natural 
language"  computer  system  and  a "natural-like"  computer  system  that  uses  a 
restricted  syntax  language  system  that  may  bear  little  resemblance  to 
"natural  Icuiguage".  Most  of  the  behavioral  and  system  advantages  favor 
the  "natural-like"  approach,  at  least  for  the  next  decade. 

Care  should  be  taken  in  generalizing  these  conclusions.  Protocols  were 
elicited  under  artificial  conditions  in  which  subects  may  have  been  concerned 
more  with  the  form  of  protocols  generated  than  with  their  effectiveness.  If 
subjects  really  had  to  use  their  protocols  to  accomplish  something  more. 
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their  choices  miqht  be  different.  The  task  settinqs  used  were  very  simple, 
much  simcler  than  the  simplest  programminq  task  domains.  Subjects  were 
initially  naive  about  the  tasks  they  were  undertaking;  users  of  a hypo- 
thetical computer  system  miqht  be  expected  to  know  more  about  what  they 
are  tryinq  to  do,  and  this  knowledge  might  bring  with  it  ideas  (perhaps 
useful,  perhaps  interfering)  about  reasonable  forms  of  expression.  These 
limitations  granted,  these  findings  may  at  least  suggest  that  questions 
about  form  of  expression  will  not  have  simple  answers. 
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FOOTNOTES 


1.  We  thank  John  Thomas,  K.  William  Scholz,  and  Stephen  Boies  for  com- 
mentin'q  on  versions  of  this  manuscript. 

2.  Now  at  Psychology  Department,  University  of  Michigan,  Ann  Arbor, 

Michigan . 

3.  Now  at  P.  . -ogy  Department,  University  of  Oregon,  Eugene,  Oregon. 

4.  Subjects  could  have  used  the  descriptons,  by  repeating  them,  as  their 
protocols  in  the  generating  conditions,  but  this  was  done  only  twice. 

5.  Sixty  percent  of  the  70  generated  Blocks  protocols  were  judged  the 
same  way  by  all  three  experimenters.  All  but  4 of  t.he  70  were  resolved 
by  majority  decision.  Two  experimenters  judged  the  70  generated  Typi.ng 
protocols  and  initially  agreed  on  80%. 

6.  Performance  wit.h  the  Restricted  Syntax  language  may  have  been  helped  by 
a practice  effect,  '-vhich  occurred  in  Blocks  but  not  in  Typing.  About 
three  times  more  Blocks  protocols  were  unambiguous-consistent  if  they 
were  the  fourth  or  fifth  ones  written  than  if  they  were  among  the 
first  three  written. 

7.  These  ambiguities  took  several  forms:  (a)  No  specification  of  position, 

e.g.,  "Next  ...  is  two  red  blocks,  then  two  yellow  blocks."  (b)  Use  of 
ambiguous  words  to  specify  position,  e.g.,  "next  to",  "beside",  "across 
from";  ("next  to"  was  usually  intended  to  mean  "to  the  right  of",  but 
scmetimes  to  mean  "above") . (c)  Variable  use  of  imprecisely  defined 

nouns,  e.g.,  "tiers",  "layers",  "lines".  (d)  Use  of  traditionally 
defined  terms  in  a non-traditional  way,  e.g.,  "row",  "column".  (e) 
Ambucruities  arising  from  more  than  one  possible  location  for  a block, 
e.g.,  "Put  a blue  block  on  top  of  red"  when  there  were  two  different 

red  blocks  with  no  blocks  on  top  of  either.  This  error  mainly  o^cuir'-: 
with  the  Restricted  Syntax  language.  (In  designing  the  block  figaie  . , 
care  was  taken  so  that  protocols  could  be  generated  without  forcing 
this  type  of  ambiguity).  ( f ) Imprecise  specification,  or  no 
specification,  of  the  relation  within  a "sub-assembly"  of  blocks,  or 
between  sub-assemblies,  e.g.,  "in  front  of  that,  using  the  same  color 
procedure  as  above,  place  three  more  blocks".  Note  that  this  example 
is  analogous  to  a DO  loop  operation,  although  neither  bound  of  the  loop 
is  specified.  Perhaps  the  reason  that  ambiguities  were  concerned  with 
spatial  position  is  because  position  demands  a reference  to  a previous 
block,  whereas  color  and  number  do  not. 

Ambiguities  in  Typing  protocols  were  due  to  (a)  linguistic  confusion, 
e.g.,  use  of  the  word  "blank"  as  an  adjective  to  specify  an  empty  cell 
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and  as  a verb  to  specify  skipping  a cell;  (b)  contradictions,  e.g.,  a 
summary  statement  about  the  entire  figure  contradicted  earlier  or 
subseguently  written  details;  (c)  unclear  specification  of  counting 
or  looping,  such  as  "go  back  to  the  beginning  of  the  line";  and  (d) 
incomplete  specifications.  In  both  tasks  subjects  tended  to  specify 
precisely  the  number  of  interations  in  a looping  operation  but  not  the 
bounds . 

8.  These  classifications  were  done  independently  by  each  experimenter,  who 
was  not  aware  under  which  instuctional  condition  each  protocol  was 
written.  In  Blocks,  81%  of  the  70  protocols  were  categorized  the  same 
way  by  all  3 experimenters,  and  all  but  two  were  resolved  by  majority 
decision.  The  agreement  among  the  three  experimenters  on  Typing  was 
low.  Fifty-nine  percent  of  the  70  protocols  were  classified  the  same 
way  by  all  three  experimenters,  and  all  but  2 were  resolved  by  majority 
decision . 

9.  The  mean  number  of  words  and  abbreviations  was  about  the  same  under  all 
instructional  constraints,  varying  from  50-70  for  blocks  and  from  35  to 
55  for  Typing.  These  rates  of  12  to  20  words  or  abbreviations  per  minute 
are  about  one-half  the  rate  of  writing  memorized  material  (30  to  40 
words  per  minute) . Procedure  protocols  typically  used  transitive  verbs 
("place",  "put",  "lay",  "build"  in  Blocks;  and  "type",  "make",  "start", 
"return"  in  Typing)  or  similar  special  constructions  ("start  with", 

"go  back  to"  in  Blocks  and  "leave  empty"  in  Typing) . Description 
protocols  typically  started  with  existential  constructions  ("There  are", 
"The  figure  contains"),  contained  passive  constructions  {"...is  filled 
with"),  or  contained  no  subject-verb  construction  ("Four  grey  blocks 
one  on  top  of  the  other...".  Some  subjects  spontaneously  reported  that 
the  Restricted  Syntax  language  made  their  task  easier  (contrary  to 
their  acutal  time  scores)  because  they  did  not  have  to  "put  things  in 
their  own  words".  Twelve  of  the  28  Blocks  Procedure  and  Description 
protocols  began  with  an  initial  general  summary  sentence,  followed  by 
specifics  about  parts  of  the  target  figures.  None  of  the  Typing 
protocols  began  with  a summary  statement.  Most  subjects  drew  some 
pictures  in  Typing  but  no  pictures  in  Blocks. 
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